AWEQ: Post-Training Quantization with Activation-Weight Equalization for Large Language Models

Large language models(LLMs) exhibit excellent performance across a variety of tasks, but they come with significant computational and storage costs. Quantizing these models is an effective way to alleviate this issue. However, existing methods struggle to strike a balance between model accuracy and hardware efficiency. This is where we introduce AWEQ, a post-training method that requires no additional training overhead. AWEQ excels in both ultra-low-bit quantization and 8-bit weight and activation (W8A8) quantization. There is an observation that weight quantization is less challenging than activation quantization. AWEQ transfers the difficulty of activation quantization to weights using channel equalization, achieving a balance between the quantization difficulties of both, and thereby maximizing performance. We have further refined the equalization method to mitigate quantization bias error, ensuring the robustness of the model. Extensive experiments on popular models such as LLaMA and OPT demonstrate that AWEQ outperforms all existing post-training quantization methods for large models

Tanshinone IIA protects against dopaminergic neuron degeneration via regulation of DJ-1 and Nrf2/HO-1 pathways in a rodent model of Parkinson’s disease

Purpose: To study the potential neuroprotective effects of tanshinone IIA, a diterpene quinone, in an experimental model of 1-methyl-4-phenyl-1, 2, 3, 6-tetrahydropyridine (MPTP)-induced Parkinson disease (PD). Methods: Mice (C57BL/6) were administered freshly-prepared MPTP at a dose of 20 mg/kg body weight intraperitoneally, 4 times at 2-h intervals, to induce PD. Doses of 12.5, 25 and 50 mg/kg tanshinone IIA were administered to the mice as treatments for PD. Pole and Rota-rod tests were carried out to assess muscular coordination and bradykinesia. Protein expressions, reactive oxygen species (ROS) and malonaldehyde and other parameters were evaluated. Results: Tanshinone IIA at doses of 12.5, 25 and 50 mg/kg reduced deficits in muscular coordination and improved learning ability of MPTP-treated mice. It also reduced loss of tyrosine hydroxylase (TH)- positive neurons following MPTP-induction. Tanshinone IIA regulated apoptotic pathway proteins, i.e., Bax and Bcl-2, and inhibited the translocation of Cyt C to the mitochondria. Oxidative stress induced by MPTP was significantly inhibited by tanshinone IIA via up-regulation of DJ-1/Nrf2 /HO-1 expression and reduction of ROS and MDA levels. Brain tissue total glutathione content was increased by tanshinone IIA treatment. Conclusion: Tanshinone IIA effectively improves antioxidant status and reduces neuronal loss following MPTP treatment. These results indicate that tanshinone IIA exerts protective effects in MPTPinduced PD in mice. Thus, tanshinone IIA has a good potential for use as a therapy for PD

Advances in multi-modal non-invasive imaging techniques in the diagnosis and treatment of polypoidal choroidal vasculopathy

Polypoidal choroidal vasculopathy (PCV) is a disease characterized by subretinal pigment epithelium (RPE) orange-red polypoidal lesions and abnormal branching neovascular networks (BNNs). In recent years, various non-invasive imaging technologies have rapidly developed, especially the emergence of optical coherence tomography angiography (OCTA), multi-spectral imaging, and other technologies, which enable the observation of more features of PCV. In addition, these technologies are faster and less invasive compared to indocyanine green angiography (ICGA). Multi-modal imaging, which combined multiple imaging techniques, provides important references for the diagnosis and treatment of PCV with the assistance of regression models, deep learning, and other algorithms. In this study, we reviewed the non-invasive imaging techniques, multi-modal imaging diagnosis, and multi-scene therapeutic applications of PCV, with the aim of providing a reference for non-invasive multi-modal diagnosis and treatment of PCV

Testing optimally weighted combination of variants for hypertension

© 2014 Zhao et al.; licensee BioMed Central Ltd. Testing rare variants directly is possible with next-generation sequencing technology. In this article, we propose a sliding-window-based optimal-weighted approach to test for the effects of both rare and common variants across the whole genome. We measured the genetic association between a disease and a combination of variants of a single-nucleotide polymorphism window using the newly developed tests TOW and VW-TOW and performed a sliding-window technique to detect disease-susceptible windows. By applying the new approach to unrelated individuals of Genetic Analysis Workshop 18 on replicate 1 chromosome 3, we detected 3 highly susceptible windows across chromosome 3 for diastolic blood pressure and identified 10 of 48,176 windows as the most promising for both diastolic and systolic blood pressure. Seven of 9 top variants influencing diastolic blood pressure and 8 of 9 top variants influencing systolic blood pressure were found in or close to our top 10 windows

A Unified Framework for HS-UAV NOMA Networks: Performance Analysis and Location Optimization

In this paper, we propose a unified framework for hybrid satellite/unmanned aerial vehicle (HS-UAV) terrestrial non-orthogonal multiple access (NOMA) networks, where satellite aims to communicate with ground users with the aid of a decode-forward (DF) UAV relay by using NOMA protocol. All users are randomly deployed to follow a homogeneous Poisson point process (PPP), which is modeled by the stochastic geometry approach. To reap the benefits of satellite and UAV, the links of both satellite-to-UAV and UAV-to-ground user are assumed to experience Rician fading. More practically, we assume that perfect channel state information (CSI) is infeasible at the receiver, as well as the distance-determined path-loss. To characterize the performance of the proposed framework, we derive analytical approximate closed-form expressions of the outage probability (OP) for the far user and the near user under the condition of imperfect CSI. Also, the system throughput under delay-limited transmission mode is evaluated and discussed. In order to obtain more insights, the asymptotic behavior is explored in the high signal-to-noise ratio (SNR) region and the diversity orders are obtained and discussed. To further improve the system performance, based on the derived approximations, we optimize the location of the UAV to maximize the sum rate by minimizing the average distance between the UAV and users. The simulated numerical results show that: i) there are error floors for the far and the near users due to the channel estimation error; ii) the outage probability decreases as the Rician factor K increasing, and iii) the outage performance and system throughput performance can be further improved considerably by carefully selecting the location of the UAV

Achievable Degrees of Freedom for the Two-Cell Two-Hop MIMO Interference Channel With Half-Duplex Relays

We consider the two-cell two-hop multiple-input–multiple-output interference channel with half-duplex relays, where each source group having $M$ single antenna users communicates with the corresponding destination with $M$ antennas via two relays, each having $M$ antennas. For such a channel, by exploiting three time slots, the previously known achievable degrees of freedom (DoF) is $2M/3$ regardless of whether the half-duplex relays have global channel state information (CSI) for the first hop. In this paper, we show that using $n\geq 3$ time slots, the achievable DoF is $(n-1)M/n$ , which is higher than the previous result of $2M/3$ DoF for the case of $n\geq 4$ . The achievability is shown by a new relaying protocol, which combines the alternate transmission strategy with an interference cancellation technique. A major implication of the derived result is that a normalized DoF of one can be achieved asymptotically without requiring global CSI at the source and relay nodes